Papers with response selection
Constructing Interpretive Spatio-Temporal Features for Multi-Turn Responses Selection (P19-1)
Copied to clipboard
| Challenge: | Existing models for response selection do not perform well when there are many candidate responses. |
| Approach: | They propose a Spatio-Temporal Matching network (STM) for response selection . they use soft alignment to obtain local relevance between context and response . |
| Outcome: | The proposed model significantly outperforms the state-of-the-art model on two large-scale multi-turn response selection tasks. |
Using Multi-Encoder Fusion Strategies to Improve Personalized Response Selection (2022.coling-1)
Copied to clipboard
| Challenge: | Existing systems that focus on persona do not explore well the correlation between persona and empathy. |
| Approach: | They propose a suite of fusion strategies that capture interaction between persona, emotion, and entailment information of the utterances. |
| Outcome: | The proposed model outperforms the previous methods by 2.3% on original personas and 1.9% on revised persona models in terms of hits@1 accuracy. |
Evaluating Dialogue Generation Systems via Response Selection (2020.acl-main)
Copied to clipboard
| Challenge: | Existing automatic evaluation metrics for open-domain dialogue systems correlate poorly with human evaluation. |
| Approach: | They propose to construct response selection test sets with well-chosen false candidates to evaluate response generation systems via response selection. |
| Outcome: | The proposed method correlates with human evaluation better than widely used metrics such as BLEU. |
TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models with self-attention encoder architectures are less useful in practice. |
| Approach: | They propose to use user and system tokens to model dialogue behavior during pre-training . they propose a contrastive objective function to simulate the response selection task . |
| Outcome: | The proposed model outperforms baseline models on four downstream tasks . it also has a few-shot ability that can mitigate the data scarcity problem . |
DS-TOD: Efficient Domain Specialization for Task-Oriented Dialog (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent work shows that self-supervised dialog-specific pretraining on large conversational datasets yields substantial gains over traditional language modeling (LM) pretraining. |
| Approach: | They propose a resource-efficient and modular domain specialization by means of domain adapters in which domain knowledge is encoded. |
| Outcome: | The proposed framework extracts domain-specific terms and then uses them to build DomainCC and DomainReddit resources based on masked language modeling and response selection objectives. |
ConvFiT: Conversational Fine-Tuning of Pretrained Language Models (2021.emnlp-main)
Copied to clipboard
Ivan Vulić, Pei-Hao Su, Samuel Coope, Daniela Gerz, Paweł Budzianowski, Iñigo Casanueva, Nikola Mrkšić, Tsung-Hsien Wen
| Challenge: | Existing Transformer-based language models (LMs) are not effective as sentence encoders when used off-the-shelf. |
| Approach: | They propose a method which turns a pretrained LM into a universal conversational encoder and task-specialised sentence encoder. |
| Outcome: | The proposed framework achieves state-of-the-art ID performance across the board with particular gains in the most challenging, few-shot setups. |
Self-training Improves Pre-training for Few-shot Learning in Task-oriented Dialog Systems (2021.emnlp-main)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models have shown promising results for few-shot learning in task-oriented dialog (ToD) systems. |
| Approach: | They propose a self-training approach that iteratively labels the most confident unlabeled data to train a stronger Student model. |
| Outcome: | The proposed approach improves state-of-the-art pre-trained models in few-shot learning scenarios for task-oriented dialog (ToD) systems when only a small number of labeled data are available. |
Do dialogue representations align with perception? An empirical study (2023.eacl-main)
Copied to clipboard
| Challenge: | masked language models produce stronger correlations than auto-regressive models, but humans and models make different response selection mistakes. |
| Approach: | They propose to use spoken conversation as a model to measure human comprehension behaviour. |
| Outcome: | The proposed model outperforms the model which produces the strongest correlation with human responses. |
ConVEx: Data-Efficient and Few-Shot Slot Labeling (2021.naacl-main)
Copied to clipboard
| Challenge: | ConVEx is an efficient pretraining and fine-tuning neural approach for slot-labeling dialog tasks. |
| Approach: | They propose an efficient pretraining and fine-tuning neural approach for slot-labeling dialog tasks that uses a pairwise cloze task and reddit data. |
| Outcome: | The proposed approach is well aligned with its intended use on slot-labeling tasks and can be used across a range of domains and data sets. |
HS-STaR: Hierarchical Sampling for Self-Taught Reasoners via Difficulty Estimation and Budget Reallocation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have incorporated reward models to guide response selection or decoding, aiming to obtain higher-quality data. |
| Approach: | They propose a Hierarchical Sampling framework for self-taught reasoners that allocates a fixed sampling budget to problem boundary-level problems and then reallocates the remaining budget toward high-utility problems during a re-sampling phase. |
| Outcome: | The proposed framework outperforms baseline models without additional sampling budgets across multiple reasoning benchmarks and backbone LLMs. |
MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understanding (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for multi-party conversation represent interlocutors and utterances individually . existing methods ignore complicated structure of MPC which may provide crucial interlocutor and tertiary semantics. |
| Approach: | They propose a pre-trained model for multi-party conversation that considers learning who says what to whom in a unified model with elaborated self-supervised tasks. |
| Outcome: | The proposed model outperforms existing models on three downstream tasks at two benchmarks. |
Do It Once: An Embarrassingly Simple Joint Matching Approach to Response Selection (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing matching models for response selection perform the independent matching (IM) approach. Existing models for matching only perform one match regardless of the number of options. |
| Approach: | They propose a joint matching approach which performs matching only once regardless of the number of options. |
| Outcome: | The proposed approach outperforms existing models and reduces training time by over half. |
Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | The performance of large language models (LLMs) on existing reasoning benchmarks has significantly improved over the past decade. |
| Approach: | They propose a benchmark dataset for evaluating the problem solving abilities of large language models (LLMs) they curate 515 challenging problems from the highly competitive IIT JEE-Advanced exam. |
| Outcome: | The proposed model performs better on open-source and proprietary models than the current model, but with techniques like self-consistency, self-refinement and chain-of-thought prompting. |
Response Selection for Multi-Party Conversations with Dynamic Topic Tracking (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing response selection methods focus on a two-party single-conversation scenario. |
| Approach: | They propose a multi-task learning framework that frames response selection as a dynamic topic tracking task to match the topic between the response and relevant conversation context. |
| Outcome: | The proposed framework outperforms existing methods on an Ubuntu IRC dataset in response selection and topic disentanglement tasks. |
Training Neural Response Selection for Task-Oriented Dialogue Systems (P19-1)
Copied to clipboard
Matthew Henderson, Ivan Vulić, Daniela Gerz, Iñigo Casanueva, Paweł Budzianowski, Sam Coope, Georgios Spithourakis, Tsung-Hsien Wen, Nikola Mrkšić, Pei-Hao Su
| Challenge: | Despite their popularity, retrieval-based models have had modest impact on task-oriented dialogue systems . main obstacle to their application is the low-data regime of most task-orientated dialogue tasks . e-commerce, banking, and other domains are applications of retrieval models . |
| Approach: | They propose a method which pretrains a retrieval-based model on large general-domain conversational corpora and fine-tunes it for the target dialogue domain. |
| Outcome: | The proposed method is evaluated on five diverse domains, ranging from e-commerce to banking. |
IRRGN: An Implicit Relational Reasoning Graph Network for Multi-turn Response Selection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on matching between candidate options and historical dialogues while ignoring the reasoning ability of the model. |
| Approach: | They propose an Implicit Relational Reasoning Graph Network to address these issues . they propose to implicitly extract dependencies between utterances and options . |
| Outcome: | The proposed model outperforms human models on two multi-turn dialogue reasoning benchmark datasets. |
Do LLMs suffer from Multi-Party Hangover? A Diagnostic Approach to Addressee Recognition and Response Selection in Conversations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Conventional evaluation methods often overlook variances in model behavior across different levels of structural complexity on interaction graphs. |
| Approach: | They propose a methodological pipeline to investigate model performance across structural attributes of conversations. |
| Outcome: | The proposed method analyzes the performance of an LLM to classify multi-party conversations . it shows that response selection relies more on the textual content of conversations compared to addressee recognition . |
GIFT: Graph-Induced Fine-Tuning for Multi-Party Conversation Understanding (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods on understanding multi-party conversations typically embed interlocutors and utterances into sequential information flows or use superficial graph structures. |
| Approach: | They propose a plug-and-play method which adapts Transformer-based pre-trained language models for universal MPC understanding. |
| Outcome: | The proposed method can adapt Transformer-based pre-trained language models for universal MPC understanding. |
The World is Not Binary: Learning to Rank with Grayscale Data for Dialogue Response Selection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to learning-to-rank response selection are suboptimal due to ignorance of diversity of response quality. |
| Approach: | They propose to use off-the-shelf response retrieval models as automatic grayscale data generators to train response selection models. |
| Outcome: | The proposed approach can be automated without human effort on grayscale data. |
DialogConv: A Lightweight Fully Convolutional Network for Multi-view Response Selection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing retrieval-based dialogue systems suffer from slow inference or huge number of parameters. |
| Approach: | They propose a lightweight fully convolutional architecture for response selection using convolution. |
| Outcome: | The proposed architecture extracts matching features of context and response from 3D views. |
UniRetriever: Multi-task Candidates Selection for Various Context-Adaptive Conversational Retrieval (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for retrieving information from a large corpus of data are sub-optimal and low efficiency. |
| Approach: | They propose a multi-task framework that functions as a universal retriever for three dominant retrieval tasks during the conversation. |
| Outcome: | The proposed framework can perform persona selection, knowledge selection, and response selection tasks simultaneously. |